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INCORPORATING CAMERA EFFECTS INTO 
EXISTING VIDEO SEQUENCES 

Background 

This invention relates generally to the editing of 
digital video and the enhancement of visual effects in 
existing video sequences. 
5 Video sequences may be captured and stored using 

digital video cameras. The digital video may be stored on 
randomly accessible media such as a hard disk drive, a 
digital video disk, or a compact disk, as examples. 

A number of video effects may be incorporated during 

10 the capture process to achieve artistic effects and to 

provide emphasis on particular features. For example, in 
the course of capturing the video, a zoom effect may be 
achieved. Without actually moving the camera, the lens 
focal length may change so that objects that are far away 

15 appear closer and vice versa. In addition, by moving the 

camera itself, a panning effect may be achieved wherein the 
viewer's point of view appears to change. 

Once the captured digital video is stored on a storage 
medium, it may be desirable to modify the video to enhance 

2 0 its presentation. In many cases, the video camera operator 
may not have thought of all the possibilities for artistic 
or creative display of the subject matter. In many cases, 
the video must be captured on the fly as events occur. 
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Thereafter, it would be desirable to inject additional 
effects, including the camera effects described above, to 
existing digital video data. 

Therefore, there is a need for a way to impose camera 
5 effects on pre-existing video sequences. 

Brief Description of the Drawings 
Figure 1 is a view of a series of frames forming a 
video sequence in which a panning effect has been 
incorporated into an existing video sequence in accordance 
10 with one embodiment of the present invention; 

Figure 2 is a view of a series of frames forming a 
video sequence in which a zoom effect has been incorporated 
into an existing video sequence in accordance with one 
embodiment of the present invention; 
15 Figure 3 is a view of a series of frames in which both 

panning and zoom effects have been incorporated into an 
existing video sequence in accordance with one embodiment 
of the present invention; 

Figure 4 is a sequence of four identical video frames 
20 of the letters A-P in a grid- like array to show how panning 
and zoom effects are incorporated into an existing video 
sequence in accordance with one embodiment of the present 
invention; 

Figure 5 shows the panning and zoom effects that 
25 result from the user selections indicated in connection 
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with Figure 4 in accordance with one embodiment of present 
invention; 

Figure 6 is a flow chart for software implementing one 
embodiment of the present invention; and 
5 Figure 7 is a block diagram of hardware in accordance 

with one embodiment of the present invention. 

Detailed Description 
A video sequence 11, shown in Figure 1, includes any 
number of individual frames such as the frames 10, 12, 14 

10 and 16. The sequence 11 may be a conventional video 

sequence that may be displayed as a storyboard having a 
series of thumbnail (reduced size) frames on a single 
display screen. Moreover, the frames 10-16 may be 
representative frames for a video sequence made up of a 

15 large number of frames. In one embodiment, representative 
frames within a larger video sequence may be selected at 
regular intervals over a sequence of selectable length and 
starting frame. 

Windows 18 and 20 are overlaid on the frames 10 and 

20 16. The windows 18 and 20 may be sized and positioned by 
the user using conventional drawing software in accordance 
with one embodiment of the present invention. The window 
18 in the first frame 10 defines a portion of the overall 
video frame 10 which the user wishes to display full 

25 screen. Thus, the portion of the frame 10 inside the 

window 18 will effectively be zoomed to a larger size for 
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display in the completed video. The zoom may be 
accomplished by digital data manipulation techniques such 
as bit mapped scaling. 

Similarly, the window 20 on the last frame 16 of the 
5 sequence 11 is also independently sized. Again using 

conventional drawing software, an overlay may be depicted 
on the screen display to indicate a particular portion of 
the video frame 16 that will be displayed in a full screen 
format . 

10 The video sequence 11 (for illustrative purposes) may 

include four frames. The thumbnail frames 10 and 16 
constitute the beginning and the ending frames of the 
sequence 11. Intervening thumbnail frames 12 and 14 are 
representative frames between the frames 10 and 16. The 

15 user may select a portion of the first and last frame, for 
enlargement to full screen display. Software, in 
accordance with one embodiment of the present invention, 
interpolates the enlargement effect defined for the first 
and last frames over the representative intervening frames 

2 0 12 and 14 and the yet undisplayed intervening frames 
represented by the frames 12 and 14 . 

In the embodiment illustrated in Figure 1, the frames 
18 and 20 may be of the same size. However, the window 18 
may be positioned in a different area of the frame 10 

25 compared to the position of the window 20 within the frame 
16. The software interpolates the position of intervening 
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windows 22 and 24 in the intervening frames 12 and 14. It 
may also interpolate the effect over undisplayed 
intervening frames represented by the frames 12 and 14. 
The interpolation may be linear or may use second or third 
order curve fitting. A bessel or spline function may be 
utilized for example. 

In accordance with one embodiment of the present 
invention, a straight line interpolation may be used 
between the center point of the window 18 and the center 
point of the window 20. Thus, figuratively, a straight 
line is drawn between the center points of the windows 18 
and 20. Depending on the number of intervening frames, a 
number of intervening windows, equal to the number of 
intervening frames, are incorporated in an equally spaced 
pattern along the length of the figurative line. Thus, as 
shown in Figure 1, the windows 22 and 24 progressively move 
from the position indicated by the window 18 to the 
position shown for the window 20. Again, other non-linear 
interpolation techniques may be used as well. 

The effect of these progressive window positions is to 
create a panning effect in the finished video which, as 
recorded, had no such panning effect. Namely, the camera's 
angle of view appears to move, by moving the viewed region 
within the existing video frames. Thus, within the 
constraints defined by the existing video frames, an 
illusion of panning may be created. In the example of 



Figure 1, the camera appears to pan from the upper left 
portion of a depicted image to a lower right portion. Even 
though there was no actual movement of the camera that 
recorded sequence 11, the appearance of camera movement may 
be achieved. 

Referring to Figure 2, a zoom effect may be 
incorporated into video which was not originally recorded 
with this effect. Again, a series of thumbnail frames 26, 
28, 3 0 and 32 represent a sequence 25 made up of a large 
number of frames that, when displayed successively, form a 
video display. The first frame 26 has a window 34 defined 
centrally thereon of a given size and the last frame 32 has 
a window 36 of a larger size. The smaller the defined 
window, the greater the enlargement of the existing 
material within the window to form a full screen image. 
Thus, in the embodiment illustrated in Figure 2, the image 
created from frame 26 is enlarged more than the image 
created from frame 32. 

Again, a straight line interpolation may be achieved 
in the intervening frames 28 and 30 using a software 
interpolation technique in one embodiment of the invention. 
Thus, the intervening windows 3 8 and 40 become 
progressively larger as one extends from the first frame 26 
to the last frame 32. The effect is that of a zoom out 
where it appears that the camera is actually moving back 
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from the imaged object. In fact, no such zooming feature 
was incorporated in the video as originally recorded. 

Referring next to Figure 3, a sequence 41 of frames 
42, 44, 46 and 48 show the effect of combining the panning 
and zooming effects illustrated in connection with Figures 
1 and 2. In this example, a relatively larger window 50 is 
positioned in the upper left corner of the first frame 42 
and a relatively smaller window 52 is positioned in the 
lower right corner of the last frame 48. Windows 54 and 56 
are interpolated between the windows 50 and 52 in the 
intervening frames 44 and 46 and any undepicted intervening 
frames represented by the frames 44 and 46. Thus, the 
windows 54 and 56 are interpolated both in terms of window 
size and window position. In the illustrated embodiment, a 
straight line interpolation is incorporated with respect to 
the change in window position and the change in window 
size. The linear changes may be progressively applied 
based on the number of intervening frames to create a 
smooth effect in one embodiment of the invention. 

While embodiments of the present invention have been 
described with the use of linear interpolation techniques, 
other known interpolation techniques may be utilized as 
well. By receiving a user selection of an effect for a 
first and last frame of a sequence, the system may 
automatically apply any type of interpolation to the 
intervening frames to create a transition between the 



effect in the first selected frame to the effect in the 
last selected frame. 

Figures 4 and 5 show the visual effect of incorporating 
panning and zooming into previously recorded video. In 
5 Figure 4, the frame 58 corresponds to the frame 42 of Figure 
3 and the frame 64 corresponds to the frame 4 8 in Figure 3. 
A window 66, which is larger in size, is positioned in the 
upper left corner of the frame 58 and a window 72 which is 
smaller in size is positioned in the lower right corner of 

10 the frame 64. Windows 68 and 70 are interpolated into the 
intervening frames 60 and 62. 

In Figure 4, a display of the letters from A to P are 
arranged in the grid work in order to better show the 
zooming and panning effect. The effect of the window 66 is 

15 shown in Figure 5 at frame 74. The letters A, B, E and F 
are shown enlarged and in a full screen view (represented 
by a thumbnail view in Figure 5) . Similarly, interpolated 
window 68 in Figure 4 becomes the full size frame 76 of 
Figure 5. In this case, the letter F is enlarged a little 

20 bit more than the letters in the frame 58. Moreover, the 
position of the window is slightly offset along the 
diagonal between the center points of the windows 66 and 
72. Thus, the letter F appears somewhat larger than the 
letter A and the letter J (which is slightly cut off) is 

25 also shown enlarged. Next, the window 70 creates the frame 
78 in full size. The letter K in frame 78 is larger than 
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the letter F. In addition, the window 70 is offset along 
the diagonal which connects the center points of the 
windows 66 and 72 with respect to the depiction shown as 
frame 76. Finally, the window 72 results in the full 
5 screen display 8 0 of an enlarged letter P. Thus, the angle 
of view from the apparent camera changes across the display 
of frames and a zooming effect is also achieved. 

While it is illustrated in connection with video 
frames with a still background (such as a series of 
10 letters) , the present invention may be applied with equal 

validity to scenes which create moving images from frame to 
frame. Each frame still constitutes an individual still 
image, and the zooming and panning effect can be applied to 
video sequences of moving images in the same way as was 
15 illustrated in connection with unchanging images or a 
series of still images. 

Referring next to Figure 6, the software 82 for 
creating the zoom and pan effects in existing video begins 
by awaiting the user selection of an initial zoom window as 
20 indicated in diamond 84. Once the user selects the frame 
to begin the imposition of the camera effect and defines a 
window thereon using a frame- like overlay, the system 
records the initial sequence and the overlaid region. The 
imposition of the video overlay can be achieved with 
25 conventional video overlay and drawing software. 
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Next, at diamond 86, the flow awaits the selection of 
a final zoom window. Once the user defines the initial 
frame, the final frame, and the windows in each of the 
beginning and ending frames, the system may begin the 

5 interpolation between those windows as indicated in block 
88. In one embodiment of the present invention, the 
interpolation for the panning and zooming effects can be 
done in two separate steps. In the block 88, the position 
is interpolated and in block 90 the zoom size is 

10 interpolated. 

Thus, the window position is interpolated in 
intervening frames, in one embodiment of the present 
invention, by a straight line interpolation between the 
center of the window in the first frame and the center of 

15 the window defined in the last frame. Similarly, the zoom 
window size may be interpolated by calculating the area of 
the first window and the last window, and interpolating the 
area linearly across the intervening frames. 

Once the windows and beginning and ending frames have 

2 0 been assigned, and the interpolation has been completed, a 
new sequence of video frames is constructed and stored as 
indicated in block 92. If desired, the new video sequence 
may be displayed either as a new set of thumbnails or as a 
finished video. The new video sequence may be stored as a 

25 separately accessible standalone file. 
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All these video sequence effects may be accomplished 
using digital video stored on a randomly accessible storage 
media. A wide variety of such systems are available. For 
example, conventional processor-based systems such as desk 
5 top computers or lap top computers have hard disk drives 
which may store video sequences. In addition, some video 
cameras may have sufficient storage and processing 
capability to implement the interpolations described above. 
Also some appliances such as televisions may now or may 

10 eventually have the processing capability to accomplish 

these effects. Similarly, some playback devices may now or 
in the future have the capability of implementing these 
effects. For example, digital versatile disk players may 
have sufficient processing capability and memory to 

15 implement these effects. 

In accordance with one embodiment of the present 
invention, a set-top box processor-based system 100 
includes a processor 102, as shown in Figure 2. The 
processor 102 is coupled to an accelerated graphics port 

20 (AGP) chipset 104. The AGP chipset 104 is coupled between 
a system memory 106 and an accelerated graphics port 105. 
The port 105 is coupled to a graphics accelerator 106 which 
controls a display on a television receiver 110. Thus, the 
display shown in Figures 1-5 may actually be depicted on 

2 5 the television receiver display in accordance with one 
embodiment of the present invention. 
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The chipset 104 may also couple to a bus 112. The bus 
112 is connected to a conventional television tuner/capture 
card 114 which receives an input from an antenna 115 such 
as a broadcast antenna, a satellite antenna or a cable 
5 distribution connection. 

A bridge 116, coupled to the bus 112, is also coupled 
to a hard disk drive 118 which may store the software 82 as 
well as the sequence of video described previously. The 
bridge 116 may also be coupled to another bus 132 which is 
10 in turn coupled to a serial input/output (SIO) device 134. 
The device 134 receives serial inputs from an interface 
140. In one embodiment of the present invention, the 
interface 14 0 may be an infrared interface which receives 
inputs from an infrared based remote control unit 142 . 
15 Also coupled to the bus 132 is a basic input/output system 
(BIOS) 142. 

Thus, in accordance with one embodiment of the present 
invention, the selection of the windows in the various 
frames may be controlled by the user from the remote 

20 control unit 144. In particular, the creation of thumbnail 
versions of a video may be implemented in software and the 
overlay of the window 66, for example, may be done using 
conventional drawing software. By simply clicking on a 
thumbnail, a window overlay is created and then the user 

25 can use cursor controls to increase the size of the window 
and its shape. 
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While the present invention has been described with 
respect to a limited number of embodiments, those skilled 
in the art will appreciate numerous modifications and 
variations therefrom. It is intended that the appended 
claims cover all such modifications and variations as fall 
within the true spirit and scope of this present invention. 

What is claimed is: 
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1. A method comprising: 

receiving an indication of a selected portion of 
a first display frame of a sequence of video frames; 

receiving an indication of a selected portion of 
a second display frame of said sequence, said first and 
second display frames separated by intervening video 
frames; and 

automatically interpolating a difference related 
to said selected portions of said first and second display 
frames over said intervening frames. 

2. The method of claim 1 wherein automatically 
interpolating a difference includes automatically 
interpolating between the position of the selected portion 
of the first selected frame and the position of the 
selected portion of the second selected frame. 

3. The method of claim 1 wherein automatically 
interpolating a difference includes automatically 
interpolating between the size of the selected portion of 
the first display frame and the size of the selected 
portion of the second display frame. 

4 . The method of claim 1 including enlarging the 
selected portion of the first and second display frames. 
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2 
3 



5. The method of claim 1 including creating a series 
of thumbnail depictions of a sequence of video frames 
displayed together as a single display. 



1 6 . The method of claim 1 including overlaying an 

2 image of a window to create said indications of said 

3 selected portions. 

1 7. The method of claim 1 wherein automatically 

2 interpolating includes automatically linearly interpolating 

3 a difference between said first and second display frames. 

1 8. The method of claim 1 wherein automatically 

2 interpolating includes automatically creating a panning 

3 effect between said selected portion of the first display 

4 frame and the selected portion of said second display 

5 frame . 

1 9. The method of claim 1 wherein automatically 

2 interpolating includes automatically creating a zoom effect 

3 between the selected portion of the first display frame and 

4 the selected portion of the second display frame. 

1 10. The method of claim 9 including automatically 

2 creating a pan effect between said selected portion of said 
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3 first display frame and the selected portion of said second 

4 display frame. 



1 11. An article comprising a medium for storing 

2 instructions that cause a processor-based system to: 

3 receive an indication of a selected portion of a 

4 first display frame of a sequence of video frames; 

5 receive an indication of a selected portion of a 

6 second display frame of said sequence, said first and 

7 second display frames separated by intervening video 

8 frames; and 

9 automatically interpolate a difference related to 

10 said selected portions of said first and second display 

11 frames over said intervening frames. 

1 12. The article of claim 11 further storing 

2 instructions that cause a processor-based system to 

3 automatically interpolate between the position of the 

4 selected portion of the first selected frame and the 

5 position of the selected portion of the second selected 

6 frame . 



1 13. The article of claim 11 further storing 

2 instructions that cause a processor-based system to 

3 automatically interpolate between the size of the selected 
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4 portion of the first display frame and the size of the 

5 selected portion of the second display frame. 



1 14. The article of claim 11 further storing 

2 instructions that cause a processor-based system to enlarge 

3 the selected portion of the first and second display 

4 frames* 

1 15. The article of claim 11 further storing 

2 instructions that cause a processor-based system to create 

3 a series of thumbnail depictions of a sequence of video 

4 frames displayed together as a single display. 

1 16. The article of 11 further storing instructions 

2 that cause a processor-based system to overlay an image of 

3 a window to create said indications of said selected 

4 portions. 

1 17. The article of claim 11 further storing 

2 instructions that cause a processor-based system to 

3 automatically linearly interpolate a difference between 

4 said first and second display frames. 



1 18. The article of claim 11 further storing 

2 instructions that cause a processor-based system to 

3 automatically create a panning effect between said selected 
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4 portion of a first display frame and the selected portion 

5 of said second display frame. 

1 19. The article of claim 11 further storing 

2 instructions that cause a processor-based system to 

3 automatically create a zoom effect between the selected 

4 portion of the first display frame and the selected portion 

5 of the second display frame. 

1 20. The article of claim 19 further storing 

2 instructions that cause a processor-based system to 

3 automatically create a panning effect between said selected 

4 portion of said first display and the selected portion of 

5 said second display frame. 

1 21. A system comprising: 

2 a processor; and 

3 a storage coupled to said processor, said storage 

4 storing software that causes said system to receive an 

5 indication of a selected portion of a first display frame 

6 of a sequence of video frames, receive an indication of a 

7 selected portion of a second display frame of said 

8 sequence, said first and second display frames separated by 

9 intervening video frames and automatically interpolate a 

10 difference related to said selected portions of said first 

11 and second display frames over said intervening frames. 
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1 22. The system of claim 21 including a display 

2 coupled to said processor. 



1 23. The system of claim 22 wherein said storage 

2 stores a graphical user interface which displays a video 

3 sequence as a series of thumbnail frames. 

1 24. The system of claim 23 wherein said graphical 

2 user interface displays a beginning and ending frame as a 

3 thumbnail and displays representative intervening frames 

4 between said beginning and ending frame. 

1 25. The system of claim 21 wherein said software 

2 causes said system to automatically interpolate between the 

3 position of the selected portion of the first selected 

4 frame and the position of the selected portion of the 

5 second selected frame. 

1 26. The system of claim 21 wherein said software 

2 causes said system to automatically interpolate between the 

3 size of the selected portion of the first display frame and 

4 the size of the selected portion of the second display 

5 frame. 
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1 27. The system of claim 21 further storing software 

2 that enables a user to indicate a portion of a video frame 

3 to be enlarged by drawing a rectangle over said portion. 

1 28. The system of claim 27 wherein said software 

2 automatically enlarges the selected portion of the first 

3 and second display frames. 

1 29. The system of claim 21 wherein said software 

2 automatically linearly interpolates a difference between 

3 said first and second display frames. 

1 30. The system of claim 21 wherein said software 

2 automatically selectively adds a panning and zooming effect 

3 to stored video frames . 
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INCORPORATING CAMERA EFFECTS INTO 
EXISTING VIDEO SEQUENCES 

Abstract of the Disclosure 
A system may be provided which enables pan and zoom 
features to be added to existing stored video. A graphical 
user interface may display a video sequence as a series of 
5 thumbnail size frame images. The user may then indicate on 
any two such thumbnail sizes images a region which may be 
enlarged. The software then interpolates the degree of 
enlargement and the position on the enlarged region between 
the two selected frames over any intervening frames. 
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Send correspondence to Timothy N. Trop . TROP, PRUNER & HU, P.C., 8554 Katy 
Freeway, Ste. 100, Houston, TX 77024 and direct telephone calls to Timothy N. Trop, 
(713)468-8880. 

I hereby declare that all statements made herein of my own knowledge are true and that 
all statements made on information and belief are believed to be true; and further that 
these statements were made with the knowledge that willful false statements and the like 
so made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of 
the United States Code and that such willful false statements may jeopardize the validity of 
the application or any patent issued thereon. 
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Full Name of Sole/First Inventor: 
EDWARD 0. CLAPPER 


Inventor's Signature: 


Date: 


Residence: 

TEMPE, ARIZONA 


Citizenship: 
U.S. 


Post Office Address: 

101 EAST RIVIERA DRIVE, TEMPE, ARIZONA 85282 



INTL-0354 -US (P8573) 
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